Simulating a Gaussian process requires sampling from a high-dimensional Gaussian distribution, which scales cubically with the number of sample locations. Spectral methods address this challenge by exploiting the Fourier representation, treating the spectral density as a probability distribution for Monte Carlo approximation. Although this probabilistic interpretation works for stationary processes, it is overly restrictive for the nonstationary case, where spectral densities are generally not probability measures. We propose regular Fourier features for harmonizable processes that avoid this limitation. Our method discretizes the spectral representation directly, preserving the correlation structure among spectral weights without requiring probability assumptions. Under a finite spectral support assumption, this yields an efficient low-rank approximation that is positive semi-definite by construction. When the spectral density is unknown, the framework extends naturally to kernel learning from data. We demonstrate the method on locally stationary kernels and on harmonizable mixture kernels with complex-valued spectral densities.

We study a dynamic version of the implicit trace estimation problem.

In addition to its use in regression, the relationship between the sinc kernel and the classic theory is illuminated, in particular, the Shannon-Nyquist theorem is interpreted as posterior reconstruction under the proposed kernel.

Gaussian processes are flexible function approximators, with inductive biases controlledbyacovariancekernel.

We thank all the reviewers for their supportive and insightful comments. While kernel learning has now been1 broadly identified as important for good performance, the vast majority of approaches, while highly useful, focus2 on parametric methods that do not represent uncertainty over the values of the kernel, can be difficult to train, and3 difficult to specify inductive biases. In the camera ready, we will fix the typos and add in-text ref-33 erences to the figures we missed. Non-axis aligned methods are also possible35 with other generalizations of FFT (possibly [3]). Inthecameraready,wewillupdatethe40 figure to be on the count instead.9:

We show results on CIFAR-100 LT with an imbalance factor (β)of100and200.

We consider the optimization problem associated with fitting two-layers ReLU networks with respect tothesquared loss, where labels aregenerated byatarget network.

Machine learning has seen many successful achievements in recent years.